2 research outputs found
Exploring the Impact of Serverless Computing on Peer To Peer Training Machine Learning
The increasing demand for computational power in big data and machine
learning has driven the development of distributed training methodologies.
Among these, peer-to-peer (P2P) networks provide advantages such as enhanced
scalability and fault tolerance. However, they also encounter challenges
related to resource consumption, costs, and communication overhead as the
number of participating peers grows. In this paper, we introduce a novel
architecture that combines serverless computing with P2P networks for
distributed training and present a method for efficient parallel gradient
computation under resource constraints.
Our findings show a significant enhancement in gradient computation time,
with up to a 97.34\% improvement compared to conventional P2P distributed
training methods. As for costs, our examination confirmed that the serverless
architecture could incur higher expenses, reaching up to 5.4 times more than
instance-based architectures. It is essential to consider that these higher
costs are associated with marked improvements in computation time, particularly
under resource-constrained scenarios. Despite the cost-time trade-off, the
serverless approach still holds promise due to its pay-as-you-go model.
Utilizing dynamic resource allocation, it enables faster training times and
optimized resource utilization, making it a promising candidate for a wide
range of machine learning applications
SPIRT: A Fault-Tolerant and Reliable Peer-to-Peer Serverless ML Training Architecture
The advent of serverless computing has ushered in notable advancements in
distributed machine learning, particularly within parameter server-based
architectures. Yet, the integration of serverless features within peer-to-peer
(P2P) distributed networks remains largely uncharted. In this paper, we
introduce SPIRT, a fault-tolerant, reliable, and secure serverless P2P ML
training architecture. designed to bridge this existing gap.
Capitalizing on the inherent robustness and reliability innate to P2P
systems, SPIRT employs RedisAI for in-database operations, leading to an 82\%
reduction in the time required for model updates and gradient averaging across
a variety of models and batch sizes. This architecture showcases resilience
against peer failures and adeptly manages the integration of new peers, thereby
highlighting its fault-tolerant characteristics and scalability. Furthermore,
SPIRT ensures secure communication between peers, enhancing the reliability of
distributed machine learning tasks. Even in the face of Byzantine attacks, the
system's robust aggregation algorithms maintain high levels of accuracy. These
findings illuminate the promising potential of serverless architectures in P2P
distributed machine learning, offering a significant stride towards the
development of more efficient, scalable, and resilient applications